[Link] Hack The Web Without A Browser

Read: 19/1/2022 hackaday.com

It is a classic problem. You want data for use in your program but it is on a webpage. Some websites have an API, of course, but usually, you are on your own. You can load the whole page via HTTP and parse it. Or you can use some tools to “scrape” the site.

api api docs automation diy hackaday software web web browser apps web browsers web crawlers web extensions web platforms web scraping web services webdev webhooks

[Link] StormCrawler demonstrates the power of open source

Read: 15/8/2021 cameraforensics.com

We sat down with our team member Julien Nioche, resident web crawling and big data expert and the developer of the open-source project ‘StormCrawler’.

big data crime cybercrime data processing foss internet interviews software software announcements software development web web crawlers web services